Do machines dream of atoms? Crippen's logP as a quantitative molecular benchmark for explainable AI heatmaps

While there is a great deal of interest in methods aimed at explaining machine learning predictions chemical properties, it difficult to quantitatively benchmark such methods, especially for regression tasks. We show that the Crippen logP model (J. Chem. Inf. Comput. Sci. 1999, 39, 868) provides an excellent atomic attribution/heatmap approaches, if ground truth heatmaps can be adjusted reflect molecular representation. The “atom attribution from finger prints”-method developed by Riniker and Landrum 2013, 5, 43) gives are reasonable agreement with contribution most molecules, average heatmap overlaps up 0.54. increased significantly (to 0.75) when contributions match fact representation fragment-based rather than atom-based (the print-adapted (FPA) vector). Most corresponding FPA relatively insensitive training set size results close converged 1000 although molecules low overlap some change significantly. Using “remove atom” approach graph convolutional neural networks (GCNNs) suggested Matveieva Polishchuk Cheminform. 2021, 13, 41) we find 0.47 model. Like simpler benchmarks classification tasks have come before it, this work sets bar